
What is claimed is: 

1 1. A remote controller comprising structure information 

2 supply means for holding structure information for various 

3 types of appliances in advance, and portable remote control 

4 means for controlling said appliances, 

5 • (1) said structure information supply means including: 

6 a transmitter's structure information storage unit 

7 for storing the structure information, the structure 

8 information including control information for each 

9 ' appliance and object information, said control 

10 information being used by said remote control means in 

11* controlling said appliances, said object information 

12 representing manipulation objects forming a manipulation 

13 unit of said remote control means; and 

14 a structure information transmission unit for 

15 retrieving the control information and the object 

16 information from said transmitter's structure 

17 information storage unit to send the retrieved 

18 information to said remote control means, 

19 (2) said remote control means including: 

20 a structure information receipt unit for receiving 

21 the control information and the object information from 

22 said structure information transmission unit; 

23 a receiver's structure information storage unit for 

24 storing the control information and the object 

25 information received by said structure information 
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26 receipt unit, 

27 a display unit for displaying a picture having a 

28 * manipulation area used for manipulating said appliances; 
.29 .' and 

■ 30 a first display control unit for controlling said 

31? / display unit to place the manipulation objects on the 

32> manipulation area based on the object information stored 

33 in said receiver's structure information storage unit. 

1 2. The remote controller of Claim 1, wherein said 

2 remote control means further includes: 

3 a manipulation detection unit for detecting which 

4 manipulation object is manipulated on the manipulation area; 

5 a first control signal transmission unit for 

6 transmitting a control signal assigned for the manipulation 

7 object detected by said manipulation detection unit to said 

8 appliances based on the control information stored in said 

9 receiver's structure information storage unit, 

10 whereby said appliances operate in accordance with the 

11 control signal from said remote control means. 

1 3. The remote controller of Claim 2, wherein said 

2 structure information supply means is installed in each 

3 appliance controlled by the control signal from said remote 

4 control means. 
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1 4. The remote controller of Claim 3, wherein 

2 said structure information supply means further 

3 includes : 

4 a structure information generation unit for 

5 generating state-based control information depending on 

6 an action state of each appliance, said state-based 

7 control information being transmitted to said remote 

8 control means by said structure information transmission 

9 unit; 

10 an object information generation unit for 

11 generating the object information depending on the 

12 action state of each appliance, the object information 

13 being transmitted to said remote control means by said 

14 structure information transmission unit, and 

15 wherein said remote control means further includes: 

16 a generated information receipt unit for receiving 

17 the state-based control information from said structure 

18 information transmission unit; 

19 a generated information update unit for updating 

20 the control information and the object information in 

21 said receiver's structure information storage unit in 

22 . accordance with the state-based control information 

23 received by said generated information receipt unit; 

24 a second control signal transmission unit for 

25 transmitting a control signal assigned to the 

26 manipulated object detected by said manipulation 
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27 detection unit to said appliances based on updated 

28 control information stored in said receiver's structure 

29 information storage unit; 

30^ / a generated object information receipt unit for 

Si 4 receiving updated object information from said structure 

32 f t % information transmission unit; and 

33 ' a second display control unit for controlling said 

34 display unit to place updated manipulation objects on 

35 - -the manipulation area based on the updated object 

36 information received by said generated object 

37 ; information receipt unit. 

1 5. The remote controller of Claim 2, wherein said 



2 remote control means further includes a structure information 

3 request unit for sending a request to said structure 

4 information supply means to transmit the structure information 

5 and the object information for said appliances, and 

6 wherein said structure information supply means further 

7 includes a structure information request direction unit for 

8 directing said structure information transmission unit to 

9 transmit the control information and the object information for 

10 said appliances upon receipt of the request from said structure 

11 information request unit, and 

12 wherein said transmitter's structure information storage 

13 unit stores the control information in relation with the object 

14 information for each appliance. 
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1 6. A remote controller comprising structure information 

2 supply means for holding structure information for various 

3 types of appliances in advance, first remote control means for 
.4 controlling said appliances, and second portable remote control 

5 ; m£ans for controlling said appliances, 

6 (1) said structure information supply means including: 

7 a transmitter's structure information storage unit 

8 for storing the structure information, the structure 

9 information including control information for each 

10 appliance and object information, said control 

11 information being used by said second remote control 

12 means in controlling said appliances, said object 

13 information representing manipulation objects forming 

14 a manipulation unit of said second remote control means; 

15 and 

16 a structure information transmission unit for 

17 retrieving the control information and the object 

18 information from said transmitter's structure 

19 information storage unit to send the retrieved 

20 information to said second remote control means, 

21 (2) said first remote control means including: 

22 a manipulation acceptance unit for accepting a 

23 manipulation direction; and 

24 a signal transmission unit for transmitting a 

25 control signal assigned to the manipulation direction 
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26 accepted by said manipulation acceptance unit, 

27 whereby said appliances operate in accordance with the 

28 control signal from said first remote control means, 

29 (3) said second remote control means including: 

30 ; "'• a structure information receipt unit for receiving 

31 the control information and the object information from 
32. • said structure information transmission unit; 

33/ a receiver's structure information storage unit for 

34 storing the control information and the object 

35 information received by said structure information 

36 receipt unit, 

37 a display unit for displaying a picture having a 

38 manipulation area used for manipulating said appliances; 

39 a first display control unit for controlling said 

40 display unit to place the manipulation objects on the 

41 manipulation area based on the object information stored 

42 in said receiver's structure information storage unit; 

43 a manipulation detection unit for detecting which 

44 manipulation object is manipulated on the manipulation 

45 area; 

46 a first control signal transmission unit for 

47 transmitting a control signal assigned for the 

48 manipulation object detected by said manipulation 

49 detection unit to said appliances based on the control 

50 information stored in said receiver's structure 

51 information storage unit, 
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52 whereby said appliances operate in accordance with the 

53 control signal from said second remote control means. 

1 7. A synthesis apparatus including a plurality of 

2 appliances for synthesizing menu information withheld by each 

3 appliance, said menu information being hierarchical information 

4 * enabling a user to control said appliances interactively, 

5 (1) one of said plurality of appliances including: 

6 first menu storage means for storing first menu 

7 information, said first menu information being 

8 hierarchical information to be used to control said one 

9 appliance; 

10 menu information receipt means for receiving the 

11 menu information from the other appliances; 

12 menu information synthesis means for synthesizing 

13 the menu information received by said menu information 

14 receipt means and first menu information stored in said 

15 first menu information storage unit, resulting 

16 synthesized menu information being used as a menu- 

17 information selection menu at a highest position in a 

18 hierarchy; and 

19 menu display means for displaying the menu- 

20 information selection menu generated by said menu 

21 information synthesis means, 

22 (2) each of the other appliances including: 

23 menu storage means for storing the menu information 
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24 unique to each appliance, the menu information being 

25 hierarchical information to be used to control each 

26 appliance; and 

27 menu transmission means for transmitting the unique 

28 ■ ' menu information in said menu storage means to said one 
.29 appliance. 

1 8. The synthesis apparatus of Claim 7, wherein said one 

2 appliance further includes synthesis rule storage means for 

3 storing rules regulating menu-information synthesis by said 

4 menu information synthesis means, 

5 whereby said menu information synthesis means 

6 synthesizes the menu information from the other appliances and 

7 the first menu information in accordance with the synthesis 

8 rules . 

1 9. The synthesis apparatus of Claim 8, wherein said 

2 menu transmission means includes: 

3 a menu information retrieval direction unit for 

4 detecting a connection with said one appliance and for giving 

5 a direction to retrieve the menu information; 

6 a menu information retrieval unit for retrieving the 

7 menu information from said menu storage means upon receipt of 

8 the direction from said menu information retrieval direction 

9 unit; and 

10 a menu transmission unit for transmitting the menu 
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11 information retrieved by said menu information retrieval unit 

12 to said menu information receipt means. 

J '. 

1 10. The synthesis apparatus of Claim 9, wherein said 

2 mejnu information synthesis means includes: 

3 ■ a menu selection acceptance unit for accepting a 

4 selection of an appliance subject to control manipulation using 

5 "the menu-information selection menu displayed by said menu 

6 display means; 

7 a menu information retrieval unit for retrieving the 

8 menu information of the appliance accepted by said menu 

9 selection acceptance unit from said menu information synthesis 

10 means; and 

11 a menu switch control unit for changing a display on 

12 said menu display means in accordance with the menu information 

13 retrieved by said menu information retrieval unit. 

1 11. The synthesis apparatus of Claim 10, wherein said 

2 menu information synthesis means further includes: 

3 a menu information temporarily storage unit for 

4 temporarily storing the menu information received by said menu 

5 information receipt means; 

6 . a first menu information retrieval unit for retrieving 

7 the first menu information from said first menu storage means; 

8 a synthesis rule retrieval unit for retrieving the 

9 synthesis rules from said synthesis rule storage means, said 
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10 synthesis rules including a synthesis rule for a hierarchial 

11 structure, a selection rule for a displayed shape, a synthesis 

12 rule for a panel object, and a synthesis rule for control of 

13 saxd appliances; and 

14 a hierarchial structure synthesis unit for synthesizing 

15 'the menu information stored in said menu information 

16 temporarily storage unit and the first menu information 

17 retrieved by said first menu information retrieval unit in 

18 accordance with the hierarchical synthesis rule retrieved by 

19 said synthesis rule retrieval unit. 

1 12. The synthesis apparatus of Claim 9, wherein said 

2 menu information synthesis means further includes: 

3 a menu information temporarily storage unit for 

4 temporarily storing the menu information received by said menu 

5 information receipt means; 

6 a first menu information retrieval unit for retrieving 

7 the first menu information from said first menu storage means; 

8 a synthesis rule retrieval unit for retrieving the 

9 synthesis rules from said synthesis rule storage means, said 

10 synthesis rules including a synthesis rule for a hierarchial 

11 structure, a selection rule for a displayed shape, a synthesis 

12 rule for a panel object, and a synthesis rule for control of 

13 said appliances; and 

14 a hierarchial structure synthesis unit for synthesizing 

15 the menu information stored in said menu information 
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16 temporarily storage unit and the first menu information 

17 retrieved by said first menu information retrieval unit in 

18 accordance with the hierarchical synthesis rule retrieved by 

19 said synthesis rule retrieval unit. 

1 13. In a remote control system comprising a plurality 

2 of appliances and a remote controller for transmitting control 

3 signals to said plurality of appliances, 

4 (1) said remote controller including: 

5 menu information request transmission means for 

6 transmitting a menu information request to each 

7 appliance, said menu information being hierarchial 

8 information unique to each appliance for enabling a user 

9 to control each appliance interactively; 

10 menu receipt means for receiving the menu 

11 information transmitted from each appliance; 

12 menu information synthesis means for synthesizing 

13 the menu information received by said menu receipt 

14 means; 

15 menu display means for visually displaying 

16 synthesized menu information generated by said menu 

17 information synthesis means; 

18 input manipulation means for accepting a user's 

19 input manipulation to control said appliances; and 

20 control signal transmission means for transmitting 

21 the control signals to said appliances based on the 
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22 input manipulation accepted by said input manipulation 

23 means, 

24 (2) each appliance including: 

25 menu information request receipt means for 

26 receiving the menu . information request from said menu 

27 information request transmission means; 

28 menu information transmission means for 

29 transmitting the menu information to said menu receipt 

30 means when said menu information request receipt means 

31 receives the menu information request; 

32 control signal receipt means for receiving the 

33 control signals from said control signal transmission 

34 means; and 

35 ' control means for controlling said appliances as 

36 per control signals received by said control signal 

37 receipt means. 

1 14. The remote control system of Claim 13, wherein said 

2 menu information synthesis means includes: 

3 a menu information storage unit for storing the menu 

4 information of each appliance received by said menu receipt 

5 means, the menu information being hierarchical information 

6 including a panel object name and shape information; 

7 a synthesis rule storage unit for storing synthesis 

8 rules regulating menu-information synthesis, said synthesis 

9 rules including a synthesis rule for a hierarchial structure, 
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10 a synthesis rule for a panel object, a selection rule for a 

11 displayed shape, and a synthesis rule for control of said 

12 appliances; 

13 . 4 a menu synthesis unit for synthesizing the menu 

14 information of said appliances stored in said menu information 

15 storage unit into one hierarchical structure in accordance with 

16 the hierarchical synthesis rules in said synthesis rule storage 

17 unit; 

18 a panel object synthesis unit for synthesizing the menu 

19 information of said appliances in said menu information storage 

20 unit into one panel object in accordance with the synthesis 

21 rule for the panel object stored in said synthesis rule storage 

22 unit, the synthesis rule for the panel object being to 

23 synthesize the menu information identified by an identical 

24 panel object name; 

25 a shape synthesis unit for making the shape information 

26 uniform for each appliance for the menu information in said 

27 menu information storage unit in accordance with the selection 

28 rule for the displayed shape in said synthesis rule storage 

29 unit, said selection rule being to select the shape information 

30 of a specified appliance and apply the same to the shape 

31 information for each appliance for the menu information in said 

32 menu information storage unit; and 

33 an action append unit for inserting data as to an 

34 control action in accordance with the synthesis rule for the 

35 control of said appliances, the synthesis rule for the control 
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36 being to insert the data as to the control action described by 

37 the synthesis rule for the control at a position specified by 

38 the synthesis rule for the control. 

1 15. The remote control system of Claim 14, wherein said 

2 menu display means includes: 

3 a bit map data conversion unit for converting the menu 

4 information synthesized by said menu synthesis unit, panel 

5 synthesis unit, "shape synthesis unit, and action append unit 

6 into bit map data for a visual display; 

7 a display unit for displaying the menu information 

8 converted into the bit map data by said bit map data conversion 

9 unit; 

10 a first manipulation acceptance unit for accepting a 

11 user's selection of menu displayed by said display unit; and 

12 a selected menu notice unit for notifying a menu 

13 selected and accepted by said first manipulation acceptance 

14 unit to said menu information synthesis means. 

1 16. The remote control system of Claim 15, wherein 

2 said menu information synthesis means further includes a menu 

3 information retrieval unit for retrieving the menu information 

4 from said menu storage unit for the appliance corresponding to 

5 the menu notified by said selected menu notice unit, retrieved 

6 menu information including information as to manipulation 

7 objects displayed by said display unit, and 
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8 ,• wherein said menu display means further includes: 

9 a menu information conversion unit for converting 

10 the menu information retrieved by said menu information 

11 retrieval unit into the bit map data, the menu 
12; information retrieved by said menu information retrieval 
•13 unit including information as to a display of 
14 manipulation objects displayed by said display unit; 
15, a change-menu display control unit for controlling 

16 said display unit to display the menu information 

17 converted into the bit map data by said menu information 

18 conversion unit; 

19 a second manipulation acceptance unit for accepting 

20 a user's control manipulation related to a control over 

21 the appliance using the manipulation objects displayed 

22 by said display unit; and 

23 a manipulation notice unit for notifying the 

24 control manipulation accepted by said second 

25 manipulation acceptance unit to said control sinal 

26 transmission unit. 

1 17. A program receiver for displaying a graphical 

2 interactive picture by receiving a program transmitted from a 

3 program transmitter, said program receiver comprising: 

4 storage means for storing a plurality of basic picture 

5 elements in advance, said plurality of basic picture elements 

6 being figures composing the graphical interactive picture 
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7 manipulated by a user; 

8 signal receipt means for receiving a signal transmitted 

9 from said program transmitter, said signal being a multiplex 

10 signal including a program and data specifying a structure of 

11 the graphical interactive picture; 

12 signal separation means for separating the signal 

13 received by said signal receipt means into a program signal and 

14 a graphical-interactive-picture-structure specification data 

15 signal; 

16 first graphical interactive picture generation means for 

17 generating the graphical interactive picture by combining the 

18 basic picture elements stored in said storage means based on 

19 the graphical-interactive-picture-structure specification data 

20 signal from said signal separation means; and 

21 display means for displaying the graphical interactive 

22 picture generated by said first graphical interactive picture 

23 generation means. 

1 18. The program receiver of Claim 17 further 

2 comprising: 

3 interactive manipulation means for inputting 

4 manipulation to the graphical interactive picture displayed by 

5 said display means; 

6 basic action storage means for storing a content of an 

7 action for updating the graphical-interactive-picture-structure 

8 specification data; and 
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9 second graphical interactive picture generation means 

10 for retrieving the content of the action from said basic action 

11 storage means based on action information directing an update 

12 of the graphical interactive picture upon receipt of the input 

13 manipulation from said interactive manipulation means to update 

14 £he graphical-interactive-picture-structure specification data 

15 to generate an updated graphical interactive picture. 

1 19- The- program receiver of Claim 18, wherein said 

2 basic picture elements in said storage means are composed of 

3 file names identifying each basic picture element and the bit 

4 map data for each basic picture element, and 

5 wherein said first graphical interactive picture 

6 generation means includes: 

7 a receipt 'decode unit for receiving the graphical - 

8 interactive-picture- structure specification data signal from 

9 said signal separation means to decode the same; 

10 a storage unit for storing decoded graphical - 

11 interactive-picture-structure specification data from said 

12 receipt • decode unit, the graphical-interactive-picture- 

13 structure specification data being composed of a class 

14 definition including a plurality pieces of class attribute 

15 information, a panel object definition including a plurality 

16 pieces of panel object information, an action definition 

17 including a plurality pieces of action information, and a shape 

18 definition including a plurality pieces of shape information; 
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19 a first process unit for retrieving the graphical - 

20 interactive-picture-structure specification data from said 

21 storage unit, extracting the basic picture elements 

22 corresponding to the file names by referring to the shape 

23 information from said storage unit in accordance with the 

24 retrieved graphical -interactive-picture- structure specification 

25 data, and for placing the extracted basic picture elements by 

26 referring to the panel object information; and 

27 a first display control unit for controlling the display 

28 means to display the basic picture elements placed by said 

29 first process unit as the graphical interactive picture. 

1 20. The program receiver of Claim 19, wherein 

2 said interactive manipulation means includes: 

3 an input manipulation acceptance unit for accepting 

4 a user's input manipulation to the graphical interactive 

5 picture; and 

6 an interactive signal transmission unit for 

7 transmitting the input manipulation accepted by said 

8 manipulation acceptance unit to said second graphical 

9 interactive picture generation unit as an interactive 

10 signal, and 

11 wherein said second graphical interactive picture 

12 generation means includes: 

13 an interactive signal receipt unit for receiving 

14 the interactive signal from said interactive signal 
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15 transmission unit; 

16 an interactive signal interpretation unit for 

17 , interpreting the interactive signal received by said 

18 interactive signal receipt unit; 

ig a graphical-interactive-picture-structure 

20 ■. specification data update unit for retrieving a content 

21 of an action from said basic action storage means in 

22 accordance with the interactive signal interpreted by 

23 said interactive signal interpretation unit to update 

24 the graphical-interactive-picture-structure 

25 specification data in said storage unit; 

26 a second process unit for retrieving updated 

27 graphical-interactive-picture-structure specification 

28 data from said storage unit, and for extracting the 

29 basic picture elements corresponding to the file names 

30 from said storage means to place the extracted display 

31 elements; and 

•32 a second display control unit for controlling said 

33 display means to display the basic picture elements 

34 placed by said second process unit as an updated 

35 graphical interactive picture. 

1 21. The program receiver of claim 20 further comprising 

2 information transmission means for transmitting the data of 

3 the graphical interactive picture updated by said interactive 

4 manipulation means to said program transmitter. 
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1 22. The program receiver of Claim 19 further comprising 

2 information transmission means for transmitting the data of 

3 the graphical interactive picture updated by said interactive 

4 manipulation means to said program transmitter. 

1 23. The program receiver of Claim 18, wherein 

2 said basic picture elements in said storage means are 

3 composed of file' names identifying each basic picture element 

4 and the bit map data for each basic picture element, and 

5 wherein said first graphical interactive picture 

6 generation means includes: 

7 a receipt «decode unit for receiving the graphical- 

8 interactive-picture-structure specification data signal 

9 from said signal separation means to decode the same; 

10 a data division unit for dividing the decoded 

11 graphical-interactive-picture-structure specification 

12 data from said receipt • decode unit into a first 

13 graphical-interactive-picture-structure specification 

14 data and a second graphical-interactive-picture- 

15 structure specification data, said first graphical- 

16 interactive-picture- structure specification data 

17 excluding a synthesis instruction, said second 

18 graphical- interactive-picture-structure specification 

19 data including the synthesis instruction, the graphical- 

20 " interactive-picture-structure specification data being 
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21 composed of a panel object definition including a 

22 • plurality pieces of panel object information, an action 

23 definition including a plurality pieces of action 

24 information, a shape definition including a plurality 

25 pieces of shape information, and synthesis direction 

26 data including a synthesis command in case of the second 
!27 graphical-interactive-picture-structure specification 

• 28 data; 

29 a first graphical-interactive-picture-structure 

30 specification data storage unit for storing the first 

31 graphical-interactive-picture-structure specification 

32 data; 

33 a second graphical-interactive-picture-structure 

34 specification data storage unit for storing the second 

35 graphical-interactive-picture-structure specification 

36 data; 

37 a graphical interactive picture synthesis unit for 

38 synthesizing the first graphical- interactive-picture- 

39 .; structure specification data in said first graphical- 
40. interactive-picture- structure specification data storage 

41 unit and the second graphical-interactive-picture- 

42 structure specification data in said second graphical- 

43 interactive-picture- structure specification data storage 

44 unit in accordance with the synthesis command of the 

45 synthesis direction data stored in said second 

46 graphical-interactive-picture-structure specification 
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47 ■ data storage unit; 

48 a graphical-interactive-picture-strutture 

49 specification data update unit for updating the first 

50 graphical-interactive-picture-structure specification 

51 data in said first graphical-interactive-picture- 

52 structure specification data storage unit with the 

53 synthesized graphical-interactive-picture-structure 

54 specification data from said graphical interactive 

55 picture synthesis unit; 

55 a first process unit for retrieving updated 

57 graphical-interactive-picture-structure specification 

58 data from said first graphical-interactive-picture- 

59 structure specification data storage unit, and for 

60 extracting the basic picture elements corresponding to 

61 the file names by referring to the shape information in 

62 said storage means to place the extracted basic picture 

63 elements by referring to the panel object information; 

64 and 

65 a first display control unit for controlling said 

66 display means to display the basic picture elements 

67 placed by said first process unit as an updated 

68 graphical interactive picture. 

1 24. The program receiver of Claim 23 further comprising 

2 information record means for outputting data related to the 

3 graphical interactive picture as per manipulation form said 
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interactive manipulation means to make a record thereof. 

25. The program receiver of Claim 18 further comprising 
^information transmission means for transmitting the data of the 

graphical interactive picture updated by said interactive 
manipulation means to said program transmitter. 

26. The program receiver of Claim 18 further comprising 
information record means for outputting data related to the 
graphical interactive picture as per manipulation form said 
interactive manipulation means to make a record thereof. 

27. A system including peripheral appliances holding 
graphical interactive picture structure data and a display unit 
for displaying a graphical interactive picture for said 
peripheral appliances by receiving a unique manipulation 
direction signal from each peripheral appliance, said display 
unit including: 

graphical interactive picture structure data receipt 
means for receiving the graphical interactive picture structure 
data from said peripheral appliances to generate the graphical 
interactive picture; 

display's graphical interactive picture structure data 
storage means for storing the graphical interactive picture 
structure data received by said graphical interactive picture 
structure data receipt means; 
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15 graphical display element storage means for storing a 

16 plurality of graphical display elements to generate the 

17 graphical interactive picture; 

18 manipulation direction signal receipt means for 

19 receiving the manipulation direction signal from said 

20 peripheral appliances; 

21 graphical interactive picture generation means for 

22 receiving the manipulation direction signal received by said 

23 manipulation direction signal receipt means, and for retrieving 

24 the graphical interactive picture structure data from said 

25 display's graphical interactive picture structure data storage 

26 means to generate the graphical interactive picture by 

27 combining the graphical display elements stored in said 

28 graphical display element storage means; and 

29 display means for displaying the graphical interactive 

30 picture generated by said graphical interactive picture 

31 generation means. 

1 28. The system of Claim 27, wherein said graphical 

2 interactive picture generation means includes: 

3 a temporarily storage unit for temporarily storing the 

4 manipulation direction signal received by said manipulation 

5 direction signal receipt means; 

6 a retrieval unit for retrieving the manipulation 

7 direction signal from said temporarily storage unit; 

8 a graphical interactive picture structure data extract 
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9 unit for extracting the graphical interactive structure data 

10 from said display's graphical interactive picture structure 

11 data storage means in accordance with the manipulation 

12 direction signal retrieved by said retrieval unit to generate 

13 the graphical interactive picture, the graphical interactive 

14 picture structure data including object information defining 

15 objects that form the graphical interactive picture, shape 

16 information defining shapes of the objects, position 

17 information defining positions of the objects, action 

18 information defining the objects' actions that change the 

19 graphical interactive picture based on the manipulation 

20 direction signal; 

21 a graphical display element extract unit for extracting 

22 bit map data from said graphical display element storage means 

23 based on the shape information extracted by said graphical 

24 interactive picture structure data extract unit, said bit map 

25 data being stored in said graphical display element storage 

26 means in relation with the shape information; 

27 a synthesis generation unit for generating a graphical 

28 display by synthesizing the bit map data extracted by said 

29 graphical display element extract unit and an attribute value 

30 of a character string attribute of each object, said attribute 

31 value of the character string attribute displayed on the bit 

32 map data being included in the object information; 

33 a placement unit for placing the graphical display 

34 synthesized by said synthesis generation unit based oh the 

204 




35 position information in the graphical interactive picture 

36 structure data. 

1 29. The system in Claim 28, wherein said graphical 

2 interactive picture generation means further includes a 
'.-3 graphical interactive picture structure data update unit for 

4 . updating a content of the graphical interactive picture 

5 , structure data in said display's graphical interactive picture 
' 6 structure data storage unit based on the action information 

7 extracted by said graphical interactive picture structure data 

8 extract unit when the manipulation direction signal retrieved 

9 by said retrieval unit is an update direction. 

1 30. The system of Claim 29, wherein said display unit 

2 further includes: 

3 TV receiver means for converting a received program into 

4 an image; 

5 TV manipulation means for accepting a manual 

6 manipulation to said TV receiver means; 

7 TV's graphical interactive picture structure data 

8 storage means for storing the graphical interactive picture 

9 structure data related to said TV receiver means in advance; 

10 and 

11 TV's interactive picture generation means for retrieving 

12 the graphical interactive picture structure data from said TV's 

13 graphical interactive picture structure data storage means when 
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14 said TV manipulation means receives the manual manipulation to 

15 generate the graphical interactive picture by combining the 

16 graphical display elements in said graphical display element 

17 storage means to update the graphical interactive picture each 

18 time a manual manipulation is received by said TV manipulation 

19 . means. 

1 31. The system of Claim 30, wherein each peripheral 

2 appliance includes: 

3 manipulation means for accepting a user ' s action 

4 direction addressed to a self's peripheral appliance; 

5 user's graphical interactive picture structure data 

6 storage means for storing the graphical interactive picture 

7 structure data to display the graphical interactive picture as 

8 per action direction accepted by said manipulation means; 

9 control means for controlling said self's peripheral 

10 appliance to set a function by giving a direction to transmit 

11 the graphical interactive picture structure data and generating 

12 a manipulation direction signal to generate the graphical 

13 interactive picture and a manipulation direction signal to 

14 updated graphical interactive picture upon acceptance of the 

15 action direction from said manipulation means; 

16 graphical interactive picture structure ■ data 

17 transmission means for retrieving the graphical interactive 

18 picture structure data from said user's graphical interactive 

19 picture structure data storage means upon receipt of the 
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20 transmission direction from said control means, and for 

21 transmitting the retrieved graphical interactive picture 

22 structure data to said display unit; and 

*Z3 manipulation direction signal transmission means for 

24 transmitting the manipulation direction signal generated by 

25 said control means to said display unit. 

1 32. The system of Claim 31, wherein said control means 

2 includes: 

3 a first signal generation unit for generating the 

4 manipulation direction signal as per action direction from 

5 said manipulation means; and 

6 a second signal generation unit for generating a 

7 predetermined subsidiary data signal depending on current 

8 action state of said self's peripheral appliance, said 

9 subsidiary data being a supplement of the graphical interactive 

10 picture structure data and constituting an integral part 

11 thereof, said subsidiary data signal being generated by said 

12 first signal generation unit together with the manipulation 

13 direction signal. 

1 33. The system of Claim 32, wherein said manipulation 

2 means is portable. 
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34. A system comprising a remote controller for 
controlling peripheral appliances and a display unit for 
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displaying a graphical interactive picture as per manipulation 
direction signal from said remote controller, 

(1) said remote controller including: 

manipulation means for accepting an action 
direction addressed »to said remote controller; 

control means for controlling said remote 
controller to set a function upon acceptance of the 
action direction from said manipulation means, and for 
generating a manipulation direction signal to generate 
the graphical interactive picture and a manipulation 
direction signal to update the graphical interactive 
picture; 

manipulation direction signal transmission means 
for transmitting the manipulation direction signal 
generated by said control means to said display unit, 

(2) said display unit including: 

appliance's graphical interactive picture structure 
data storage means for storing the graphical interactive 
picture structure data in advance to generate the 
graphical interactive picture for said peripheral 
appliances; 

graphical display element storage means for storing 
a plurality of graphical display elements to generate 
the graphical interactive picture; 

manipulation direction signal receipt means for 
receiving the manipulation direction signal from said 
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29 remote controller; 

30 graphical interactive picture generation means for 

31 retrieving "the graphical interactive picture structure 

32 data from said appliance's graphical interactive picture 

33 structure data storage means upon receipt of the 

34 • manipulation direction signal received by said 

35 manipulation direction signal receipt means to generate 

36 the graphical interactive picture by combining the 

37 graphical - display elements stored in said graphical 

38 display element storage means based on the retrieved 

39 graphical interactive picture structure data; 

40 display means for displaying the graphical 

41 interactive picture generated by said graphical 

42 interactive picture generation means; 

43 TV receiver means for converting a received TV 

44 program into an image; 

45 TV manipulation means for accepting a manual 

46 manipulation to said TV receiver means; 

47 TV's graphical interactive picture structure data 

48 storage means for storing the graphical interactive 

49 picture structure data as to said TV receiver means in 

50 advance; 

51 TV 1 s graphical interactive picture generation means 

52 for retrieving the graphical interactive picture 

53 structure data from said TV's graphical interactive 

54 I picture structure data storage means when said TV 
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manipulation means receives the manual manipulation to 
generate the graphical interactive picture by combining 
the graphical display elements stored in said graphical 
display element storage means based on the retrieved 
graphical interactive picture structure data to update 
the graphical interactive picture each time said TV 
manipulation means receives a manual manipulation. 

35. A system comprising a relay, a remote controller, 
and a display unit, said relay being a peripheral appliance for 
transferring an action direction signal from said remote 
controller to said display unit, said relay's function being 
set by said remote controller, said display unit displaying a 
graphical interactive picture upon receipt of the signal from 
said relay, 

(1) said remote controller including: 

first manipulation means for accepting an action 
of said remote controller; 

first graphical interactive picture structure data 
storage means for storing graphical interactive picture 
structure data in advance to generate the graphical 
interactive picture as per action direction; 

first control means for controlling said remote 
controller to set a function upon receipt of the action 
direction from said first manipulation means by giving 
a direction to transmit the graphical interactive 
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picture structure data and generating a manipulation 
direction signal to generate the graphical interactive 
picture and a manipulation direction signal to update 
the graphical interactive picture; 

first graphical interactive picture structure data 
transmission means for retrieving the graphical 
interactive picture structure data from said first 
graphical interactive picture structure data storage 
means upon receipt of the transmission direction from 
said first control means to transmit the same to said 
relay; and 

first manipulation direction signal transmission 
means for transmitting the manipulation direction signal 
generated by said first control means to said relay, 
(2) said relay including: 

second manipulation means for accepting an action 
direction addressed to said relay; 

user's graphical interactive picture structure data 
storage means for storing the graphical interactive 
picture structure data in advance to generate the 
graphical interactive picture as per action direction 
addressed to said relay; 

second control means for controlling said relay 
to set a function upon receipt of the action direction 
from said second manipulation means by giving a 
direction to transmit the graphical interactive picture 
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45 structure data and generating a manipulation direction 

46 signal to generate the graphical interactive picture and 

47 a manipulation direction signal to update the graphical 

48 .■ interactive picture; 

49 graphical interactive picture structure data 

50 transmission means for retrieving the graphical 

51 interactive picture structure data from said user's 

52 graphical interactive picture structure data storage 

53 means upon receipt of the transmission direction from 

54 said second control means to transmit the same to said 

55 display unit; 

56 second manipulation direction signal transmission 

57 means for transmitting the manipulation direction signal 

58 generated by said second control means to said display 

59 unit; 

60 data relay means for receiving the graphical 

61 interactive picture structure data from said first 

62 graphical interactive picture structure data 

63 transmission means to transmit the same to said display 

64 unit; and 

65 signal relay means for receiving the manipulation 

66 direction signal from said first manipulation direction 

67 signal transmission means to transmit the same to said 

68 display unit, 

69 (3) said display unit including: 

70 graphical display element storage means for storing 
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a plurality of graphical display elements to generate 
the graphical interactive picture; 

graphical interactive picture structure data 
receipt means for receiving the graphical interactive 
picture structure data to generate the graphical 
interactive picture from said relay; 

display's graphical interactive picture structure 
data storage means for storing the graphical interactive 
picture structure data received by said graphical 
interactive picture structure data receipt means; 

manipulation direction signal receipt means for 
receiving the manipulation direction signal from said 
relay; 

graphical interactive picture generation means for 
retrieving the graphical interactive picture structure 
data from said display's graphical interactive picture 
structure data storage means upon receipt of the 
manipulation direction signal received by said 
manipulation direction signal receipt means to generate 
the graphical interactive picture by combining the 
graphical display elements stored in said graphical 
display element storage means based on the retrieved 
graphical interactive picture structure data, and for 
updating the graphical interactive picture each time 
said manipulation direction signal receipt means 
receives a manipulation direction signal; and 
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97 display means for displaying the graphical 

98 interactive picture generated by said graphical 

99 interactive picture generation means. 

1 36. A system comprising a relay, a remote controller, 

2 and a display unit, said relay being a peripheral appliance for 

3 transferring an action direction signal from said remote 

4 controller to said display unit, said relay's function being 

5 set by said remote controller, said display unit displaying a 

6 graphical interactive picture upon receipt of the signal from 

7 said relay, 

8 (1) said remote controller including: 

9 first manipulation means for accepting an action 

10 direction addressed to said remote controller; 

11 first control means for controlling said remote 

12 controller to set a function by giving a direction to 

13 transmit graphical interactive picture structure data 

14 to said relay and generating a manipulation direction 

15 signal to generate a graphical interactive picture and 

16 to update the graphical interactive picture; and 

1? first, manipulation direction signal transmission 

18 means for transmitting the manipulation direction signal 

19 generated by said first control means to said relay, 

20 (2) said relay including: 

21 second manipulation means for accepting an action 

22 direction addressed to said relay; 
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23 user's graphical interactive picture structure data 

24 storage means for storing the graphical interactive 

25 picture structure data in advance to display the 

26 i graphical interactive picture as per action direction 

27 s addressed to said relay; 

28 first graphical interactive picture structure data 

29 storage means for storing in advance the graphical 

30 interactive picture structure data to display the 

31 graphical- interactive picture as per graphical- 

32 interactive-picture-structure data transmission 

33 direction from said remote controller; 

34 second control means for controlling said relay 

35 to set a function upon receipt of the action direction 

36 from said second manipulation means by giving a 

37 direction to transmit the graphical interactive picture 

38 structure data and generating a manipulation direction 

39 signal to generate the graphical interactive picture and 

40 to update the graphical interactive picture; 

41 graphical interactive picture structure data 

42 transmission means for retrieving the graphical 

43 interactive picture structure data from said user ' s 

44 graphical interactive picture structure data storage 

45 means and said first graphical interactive picture 

46 structure data storage means upon receipt of the 

47 transmission direction from said second control means 

48 and said remote controller respectively to transmit the 
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same to said display unit; 

second manipulation direction signal transmission 
means for transmitting the manipulation direction signal 
generated by said second control means to said display 
unit; and 

signal relay means for receiving the manipulation 
direction signal received by said remote control means 
to transmit the same to said display unit, 
(3) said -display unit including: 

graphical display element storage means for storing 
a plurality of graphical display elements to generate 
the graphical interactive picture; 

graphical interactive picture structure data 
receipt means for receiving the graphical interactive 
picture structure data to generate the graphical 
interactive picture from said relay; 

display's graphical interactive picture structure 
data storage means for storing the graphical interactive 
picture structure data received by said graphical 
interactive picture structure data receipt means; 

manipulation direction signal receipt means for 
receiving the manipulation direction signal from said 
relay; 

graphical interactive picture generation means for 
retrieving the graphical interactive picture structure 
data from said display's graphical interactive picture 
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75 structure data storage means upon receipt of the 

76 manipulation direction signal received by said 

77 manipulation direction signal receipt means to generate 
• m 78 the graphical interactive picture by combining the 

79 • graphical display elements stored in said graphical 

80 display elements based on the manipulation direction, 

81 and for updating the graphical interactive picture each 

82 - : time said manipulation direction signal receipt means 

83 receives .a manipulation direction signal; and 

84 display means for displaying the graphical 

85 interactive picture generated by said graphical 

86 interactive picture generation means. 
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